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Talk Outline 


This morning: 
* Goals and requirements 
* Change units, versions, knowledge 
* Update propagation 
* Conflict detection and resolution 


This afternoon: 
* Filtered sync 
* Meta-data cleanup 
* Open Issues 


Goal: Replication of WinFS Folders 


Replica B 
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WinFS Synchronization Requirements 


Allow multi-master updates 

Support peer-to-peer topologies 

Operate at the level of WinFS Schema 

Permit configuration of "what" and "when" to sync 

Detect and handle conflicts during synchronization 

Accommodate varying network characteristics: 

intermittent connections, slow networks, offline 

* Provide out-of-the-box "zero admin" manageability, 
resilience, and ease of use 

* Guarantee convergence, even in "unmanaged" 

environments 


CC ei e UR 
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WinFS Synchronization — Approach 


Read-anywhere/update-anywhere model 

Pair-wise reconciliation between replicas 

User-defined topology and schedule 

"Net Changes" of WinFS items/folders are 

conveyed, not individual actions 

* Adapters for replication with non-WinFS machines, 
services, and devices (not discussed in this talk) 


oe ee 


Items vs. Change Units 


+ Item = granularity of consistency 
+ Change unit = granularity of update, propagation, 
and conflict detection 


* elements or fields of elements 


Item E 
y | AN 3 change 
f Pime units will be 
sentin this 
Ejment] Element] [Element] [Element] | syne session 


[Element | ("Element | 
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Versions of Changes 


* Each change has a globally-unique id (version) 
+ Version = ID of the replica making the change + 


replica-specific number 
+ Repl D 
+ Repl 


s ever-increasing at the replica 


08 2 
+ Version of change unit is version of last change 
adhe e that element 


g 
. Versions only used by synchronization protocol; 
not visible to WinFS users 


| Knowledge for Update Propagation 
+ Knowledge = concise description of the set of 
changes received by a replica 


* knowledge dapands only on local database 
* not pair-specific: not "what | have received from you 


. When enumerating changes 


e units and tombstones not covered by requester's 


changes, specify 


| Knowledge Abstractly 


Main operations on knowledge: 
+ Test if a ven knowledge covers a given change 
. Add ae a ge version to one’ s knowledge 
| Just learned of à new change 
* Merge one knowledge with another to produce 
combined knowledge 


| knew this and he knew that. and now we both know this. 


Knowledge Implementation 


Version vector = set of (replica GUID, replica- 
specific adu pairs 

Semantics: "all versions authored by this replica 
up to this number" 

+ eg shorthand fi 4 Y1 
Exceptions needed for UICE received out-of- 
order 


* eg X4 


Includes every CDS that ever made a change 


Protocol Using Knowledge 


je ine A | Machine B 
iA E 
jy 83 ! 54 
k C$ LE 
| EA kasa 
| 2 
Knowledge: Knowledge. 
| A4B3C6 D0 A3 B6 C4 D2 
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Protocol Using Knowledge 


Machine A ] Machine B 
i Ad requestChanges 22 
j | „base: Ad B3 C6 DO EE 
k ————— is D2 
mh } KES) 
k A2 
Knowledge. Knowledge: 
A4 B3 C6 DO A3 B6 C4 D2 
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Protocol Using Knowledge 


Machine A | Machine B. 
i Ad fequestChanges = 
j 83 base: Ad B3 C6 DO l eM 
k C5 ah" i 2 
| 43 k B54 
Ca rA 
conveyChanges 
| Knowledge lo 1 Knowledge: 
| A4B3C6DO A3 Bô C4 D2 
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Protocol Using Knowledge 


Machine & 


Machine B. 
ko Ad | fequestChanges = 
j 02 ] base: A4 B3 C6 DO I C4 
mo 1777 P L 
k 42 ke 
conveyChanges 
Knowledge: pod 4 Knowledge. 
bale loded A3 B6 C4 D2 
+02 
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chine ] Machine B 
| Ad fequestChanges: 


base: Ad B3 C6 DO i B 
| E | MENT 
| k A2 
conveyChanges 
| Knowledge r 7 Knowledge: 
| A4B3C6DO A3 B6 C4 02 
+02 
conveyChanges 
bundle. 1 
K BSR 
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Protocol Using Knowledge 
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[ Madine P 
i Aa requestchanges a 
j 02 | base: A4 B3 C6 DO I ca 
k gás €———— is L D A 
k AQ | 
|: A2 
conveyChanges 
Knowledge IS Knowledge 
A4 B3 C6 DO A3 B6 C4 D2 
+D2 
4 
ae conveyChanges 
bundle. 1 
K BSS 
completeBundle 
bundle: 1 
messages: 2 
learned: A3 B8 C4 D2 


Sync Protocol : Things Not Shown 


. Conflict detection and resolution 
g concurrent up: 
ing or automatically 


. Topology optimizations - 
* Read-only and slave rep 


+ Filtering 


y certain items as wall as only certain por 


| Conflict Definition 


* Two changes conflict iff 


(1) the semantics of the changes are incompatible 


(2) the changes were performed concurrently 
(without knowledge of each other) 


Conflict Detection using Knowledge 


+ "Made with" knowledge = knowledge of replica 
when change was accepted 


Stored and propagated with each change (conceptuall 
* eg. ic B5- A2 B4 Cê D1 
ade ith Kd edge 


* Changes C, and C. are concurrent iff 
(1) C,.version not in C..madeWithKnowledge and 
(2) C; version not in C,.madeWithKnowledge 


Protocol with Conflict Handling 


Machine B. 


ing A 
i S, EOD i; C4- AQ B2 C2 DO 
$ B3- A1 B2C2 D0 J: D2 -A3 B5 C4 D1 
K C5 - A2 B3 C4 DO k B$-A384C4 D2 
L A2-A1B1C1 D0 I: A2- BT C1 DO 
Knowledge: Knowledge: 
| A4 B3 C6 DO A3 B6 C4 02 
| 
Conflict Log: Conflict Log: 


Protocol with Conflict Handling 


Machine B. 


hing A. 
y, MPR o ca Do reduestChanges, i: C4- A0 B2 C3 DO 
J; B3-A1B2C2D0 „bass: Ad B3 C6 DO j| D2- A3 B5 C4 D1 
k: O5- A2 B3 C4 D0- T* k8$-A3B4C4 D2 
L A2-APBICIDO | I A2- A1 B1 C1 DO 
Knowledge: Knowledge: 
A4 B3 C6 DO B A3 86 C4 D2 
fonte Log Conflict Log: 


Protocol with Conflict Handling 
Machine B. 


réquestChanges. i; C4- A0 B2 C3 DO. 


base: A4 B3 C6 DO j A3 B5 C4 D1 
T* kB$-A3B4 CA D2 
L A2-ATBICIDO l A= 41 B1 C1D0 
conveyChanges 
y 4 
nowledge: ordi DM Knowledge: 
A4 B3 C6 DO j D2 A3 B6 C4 D2 
Conflict Log: 


‘onflict Log: 


Protocol with Conflict Handling 


Rua = i 
y HRES 82 ca po requestChanges NU HM 
| f B34At82C2DO base: A4 B3 C6 DO j D2-A3B5C4D1 
I k C5- A2 B3 C4 DO: T* kB$-A3B4 C4 D2 
L A2-ATBICIDO T. I A2- A1 B1 C1 DO 
conveyChanges 
madeWith: A3 BS C4 D1« 
Knowledge. bundle: 1 usen 
| A4 B3 C6 DO j D2 AI B6 C4 D2 
Conflict Log Conflict Log: 


Protocol with Conflict Handling 


[ hing A. 
r VP B: ca po 
j 02-A3B5C4DI 
k C5- A2 B3 C4 DO. 
L A2-A1B1 C1 D0 

| 

Knowledge: 

A4 B3 C6 DO 
+02 


| 
Conflit Log 
| 
| 
| 


requestChanges. 
base: A4 B3 C6 DO 


conveyChanges 
madeWith: A3 BS C4 D1« 
bundle: 1 
j 2 


Machine B 
i: C4 - A0 B2 C3 DO 
J D2 - A3 B5 C4 D1 


T* kB$-4A384C4 D2 


i A? = 41 B1€1D0 


Knowledge: 
A3 B6 C4 02 


Conflict Log: 


Protocol with Conflict Handling 


Lt cao requestChanges NU pa 
D2 = A3 B5 C4 D1 base: A4 B3 C6 DO j D2-A3B5C4D! 
ki C5* A2B3 C4 DO- T* kB5-A3B4C4 D2 
L A2-A1B1 C1 00 I: A2-2A1 BI C1 DO 
conveyChanges 
A: 4 
Knowledge: eb CRANI Knowledge: 
A4 B3 C6 DO i j.D2 A3 BG C4 02 
| +02 — 
| conveyChanges 
madeWith. A3 B4 C4 D2 
bundle: 1 
k B5 
‘onflict Log: Conflict Log: 


Protocol with Conflict Handling 


y, RSS Bo ca po requestChanges aN ZUM 
Jj D2-A3B5CA4D! base: A4 B3 C6 DO J D2-A3B5C4D1 
k: C5-A2B3C4 DO T* KBS-A3B4C402 
I: A2-AT1B1C1DO |; A2-A1B1C100 
conveyChanges. 
A3B5C4 
Knowledge: de er SOS ADT Knowledge: 
5 2 j.D2 A3 Bà C4 02 
*D2 
5 
| ss conveyChanges 
madeWith. A3 B4 C4 D2 
| bundle: 1 
| k B5 
Eonfiet Log: Conflict Log: 
k B5 - A3 B4 C4 DE 
completeBundle 
bundle: 1 
messages 2 


learned. A3 B6 C4 02 


learned; A3 B6 C4 D2 


Protocol with Conflict Handling 


Machine B. 


i Mashing 82 c4 D0 requestChanges. i C4- A0 B2 C3 DO 
J; 02 - A3B5C4 D1 base: A4 B3 C6 DO j| D2-A3B5C401 
k: C5 -A2 B3 C4 DO: T* kB5-A384C4 D2 
L A2-A1BTCLDO k 42-A181C100 
L conveyChanges 
leWith: A: ZI 
inowledge: leó 1 genia Knowledge: 
Ad B6 C6 D2 j 02 A3 B6 C4 02 
conveyChanges 
madeWith: A3 B4 C4 D2 
bundle: 1 
É k B5 
‘onflict Log: Conflict Log; 
k B5-4A3BA C4 DD 
V completeBundle 
K AS-AdBECGO2 bundle: 1 
messages 2 


Protocol with Conflict Handling 


y. Maching Bo ca po requestChanges Ecl Ps po 
p HZ - A3 B5 CA Dt base: A4 B3 C6 DO j D2-A3B5C4D1 
k A5-A4B6C6D2 T* KB5-A3B4C402 
L A2-A1B1 C1 D0 I: A27 A1 B1 C1 D0 
conveyChanges 
madeWith: 43 B5 C4 D1 
Knowledge: bundle: 1 basen a 
: j D2 A3 B6 C4 D2 
conveyChanges 
madeWith. A3 B4 C4 D2 
| bundie: 1 
| k B5 
Conflict Log: Conflict Log: 
| k B5- A3 BACA DE 
completeBundle 
bundle: 1 
messages 2 
learned. A3 B6 C4 D2 


Protocol with Conflict Handling 


i MARRS B: capo | ^ requestchanges N HMM 
j| 02-A3B5C4D! base: A4 B3 C6 DO ji D2-A3B5C4D! 
k: C5 +A2 B3 C4 DO 1? kB$-A3B4C4 D2 
I: A2-A1BTCLDO I: 427 41B1€1D0 
conveyChanges 
madeWith: A3 B5 C4 D1 
nowledge: bundle: 1 e 
A4 B6 C6 D2 J. o2 A3 B6 C4 D2 
| conveyChanges 
| madeWith. A3 B4 C4 D2 
| bundle: 1 
k B5 
conflict Log: Conflict Log: 
k BÉ A3 Bd C4 DD 
completeBundle 
bundle: 1 
messages 2 


learned. A3 B6 C4 D2 


| Protocol with Conflicts: Not Shown 


. Automatic Conflict Resolution 


. iher replicas 


+ Storage Optimization 
* Do not need to made with kno 
conflict! 


ledge "unless" an item is in 


* Use current base knowledge as made with knowledge 


Protocol with Implicit "Made With" Knowledge 


i Machine A fequestzhangek p ayetine B 

j 83 base: A4 B3 C6 DO T L D2 

k C5 a. 2-7 

k A2 k AQ 
Knowledge: Knowledge: 

A4 B3 C6 DO L A3 B6 C4 02 

rna Log: Conflict Log: 
| 


| 
LZ 


Protocol with Implicit "Made With" Knowledge 


Machine A Machine B 

requestChanges i C4 

ER ier base: A4 B3 C6 DO j 02 

c5 NA 9 Té KBB 

A2 Io I: A2 

conveyChanges 
4 “4 024 
Knowledge: sti desde Knowledge: 
Ad B3 C6 DO LEZ *A3 BG C4 02 
| 

Conflict Log: Conflict Log: 


| i Machine A 
f D2 


j: 
k oor 
k A2 


| 

Knowledge: 

A4 B3 C6 DO 
*D2 


| 
conflict Log 
k BE 


Protocol with Implicit "Made With" Knowledge 


Machine B 
requestChanges. i C4 
base: A4 B3 C6 DO j 02 
ar Le kB5 
I: A2 
conveyChanges 
madeWith: A3 B6 C4 D2 
bundle: 1 NNNM: 
j 02 A3 BG C4 D2 
conveyChanges 
madeWith. A3 B6 C4 D2 
bundle: 1 
k B5 
Conflict Log: 


Protocol with Implicit "Made With" Knowledge 


i Machine A raquesichanges t Paa B 
ro base: A4 B3 C6 DO [o 
k O5- A4 B3 C6 DOWD? T KBB 
| tA t 42 
| conveyChanges 
A 402 
Knowledge: mee sess: Knowledge: 
| A4 B3 C6 DO j 02 AJ BG C4 D2 
*D2 bi — ——MHÓ— TE 
conveyChanges 
| madeWith. A3 B6 C4 D2 
| bundle: 1 
| k B5 
Conflict Log: ‘Conflict Log; 
k B5- A3 BACA DE 


Protocol with Implicit "Made With" Knowledge 


i Machine A raquestChangea i E B 
yo base: Ad B3 C6 DO hk DZ 
| k C5 = 44 3 C6 DOWD? T kB5 
k A2 k AQ 
| conveyChanges. 
Knowledge: O BO GA DA Knowledge: 
Ad B6 C6 D2 j D2 A3 B6 C4 D2 
D2 
us conveyChanges 
madeWith. A3 B6 C4 D2 
bundle: 1 
| k B5 
conflit Log Conflict Log: 
k B5 - A3 B4 C4 DP. 
completeBundle 
bundle: 1 
messages 2 


learned; A3 B6 C4 D2 
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Protocol with Implicit "Made With" Knowledge 


| Machine A faquesihanges į Mesrine 
: D2 i 


| tc 
base: A4 B3 C6 DO jo 

-A4 B3 C6 DOWD? T kes 
A2 k A2 


conveyChanges 
rA 402 
Knowledge: er Ni R Knowledge: 


A4 B6 C6 D2 LEZ A3 B6 C4 D2 


conveyChanges 
| madeWith. A3 B6 C4 D2 
| bundle: 1 
k B5 


Conflict Log: Conflict Log: 


k B5- A3 B4 C4 DR 
| completaBundla 
K AS bundle: 1 

messages; 2 
learned: A3 B6 C4 D2 


Protocol with Implicit "Made With" Knowledge 


+ Machine A 
i bs 
k: AS 
L A 


Knowledge: 
AS B6 Có D2 


| 
| 
cone Log: 
| 
| 


reguesiChanges 
base: Ad B3 C6 DO 


conveyChanges 
madeWith: 43 B6 C4 02 
bundle: 1 
j b2 


conveyChanges 
madeWith. A3 B6 C4 D2 
bundle: 1 
k B5 


completeBundle 
bundle: 1 
messages: 2 


learned. A3 B6 C4 D2 


Machine B 


t C4 
j D2 
T KBB 
I: A2 


Knowledge: 
A3 B6 C4 D2 


Conflict Log: 


L 
| 
i 
EE 


Protocol with Implicit "Made With" Knowledge 


i Machine Mahiwa — 


D2 


n 


H 
k 
| 


nowledge: 
AS B6 Co D2 


‘onflict Log: 


Machine B. 
requestChanges. | C4 


base: Ad B3 C6 DO 


i 
jo 
T KBB 
tL A2 
conveyChanges 
madeWith: 43 B6 C4 D2 : 
bundle: 1 Knowledge! 
| D2 A3 B0 C4 02 


conveyChanges 
madeWith. A3 B6 C4 D2 
bundie: 1 
k B5 


Conflict Log: 


completeBundle 
bundle: 1 
messages 2 
learned. A3 B5 C4 D2 


Summary 


Key features of WinFS Synchronization: 

* Favors availability over consistency 

* Achieves robustness and scalability through 
pair-wise, knowledge-driven protocol 

* Makes efficient use of bandwidth and storage 

* Allows arbitrary communication topology 

* Guarantees eventual convergence if well- 
connected topology 

* Detects conflicts; permits manual or 
automatic resolution 


